Skip to content

Conversation

@weboko
Copy link
Collaborator

@weboko weboko commented Aug 20, 2025

Problem / Description

For Send API that is part of Waku API to work we need a way to setup subscription when message is getting sent so that we could detect Filter and Store message loss.

For this Encoder is not sufficient.

Solution

We should introduce new entity instead of Encoder and Decoder that will combine their functionality and can be used in Filter and LightPush at the same time.

Notes


Checklist

  • [ ] Code changes are covered by unit tests.
  • [ ] Code changes are covered by e2e tests, if applicable.
  • [ ] Dogfooding has been performed, if feasible.
  • [ ] A test version has been published, if required.
  • All CI checks pass successfully.

@github-actions
Copy link

github-actions bot commented Aug 20, 2025

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
Waku node 70.51 KB (+0.12% 🔺) 1.5 s (+0.12% 🔺) 1.8 s (-18.13% 🔽) 3.2 s
Waku Simple Light Node 121.89 KB (+0.03% 🔺) 2.5 s (+0.03% 🔺) 1.8 s (+37.34% 🔺) 4.3 s
ECIES encryption 20.82 KB (+0.11% 🔺) 417 ms (+0.11% 🔺) 286 ms (-57.73% 🔽) 702 ms
Symmetric encryption 20.22 KB (-0.4% 🔽) 405 ms (-0.4% 🔽) 602 ms (+41.03% 🔺) 1.1 s
DNS discovery 51.69 KB (+0.24% 🔺) 1.1 s (+0.24% 🔺) 1.4 s (+76.16% 🔺) 2.4 s
Peer Exchange discovery 52.22 KB (-0.15% 🔽) 1.1 s (-0.15% 🔽) 1.3 s (+44.37% 🔺) 2.4 s
Peer Cache Discovery 46.04 KB (-0.14% 🔽) 921 ms (-0.14% 🔽) 952 ms (+29.79% 🔺) 1.9 s
Privacy preserving protocols 53.61 KB (+0.14% 🔺) 1.1 s (+0.14% 🔺) 1.3 s (+17.66% 🔺) 2.4 s
Waku Filter 55.38 KB (+0.11% 🔺) 1.2 s (+0.11% 🔺) 919 ms (-15.65% 🔽) 2.1 s
Waku LightPush 52.94 KB (+0.06% 🔺) 1.1 s (+0.06% 🔺) 1.4 s (+17.69% 🔺) 2.5 s
History retrieval protocols 59.3 KB (+0.04% 🔺) 1.2 s (+0.04% 🔺) 958 ms (-18.11% 🔽) 2.2 s
Deterministic Message Hashing 28.36 KB (-0.17% 🔽) 568 ms (-0.17% 🔽) 702 ms (+0.25% 🔺) 1.3 s

@weboko weboko requested a review from Copilot August 20, 2025 15:31
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new Codec entity that combines the functionality of Encoder and Decoder to support the Send API's requirement for setting up subscriptions during message sending to detect Filter and Store message loss.

  • Adds new ICodec interface that extends both IEncoder and IDecoder
  • Creates Codec class implementation with createCodec factory function
  • Marks existing createEncoder and createDecoder methods as deprecated in favor of the new codec approach

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/interfaces/src/message.ts Defines the new ICodec interface type
packages/interfaces/src/waku.ts Adds CreateCodecParams type and createCodec method to IWaku interface, deprecates encoder/decoder methods
packages/core/src/lib/message/codec.ts Implements the Codec class and createCodec factory function
packages/core/src/lib/message/constants.ts Extracts shared constants OneMillion and Version
packages/core/src/lib/message/version_0.ts Updates to use extracted constants
packages/core/src/lib/message/index.ts Exports new codec functionality and constants
packages/core/src/index.ts Exports codec types and factory function
packages/sdk/src/waku/waku.ts Implements createCodec method in WakuNode class
packages/rln/src/message.ts Updates to use extracted Version constant
Comments suppressed due to low confidence (1)

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@weboko weboko marked this pull request as ready for review August 20, 2025 20:46
@weboko weboko requested a review from a team as a code owner August 20, 2025 20:46
@weboko weboko requested a review from fryorcraken August 20, 2025 20:46
@fryorcraken
Copy link
Collaborator

It seems to me that you need to extend the Encoder instead of creating a new class, no? Also, not sure you need to decode messages received via filter to confirm that they are the one you sent.

Instead, you "just" need to be able to easily calculate message hash so you can look monitor whether a given message hash (of an outbound message) is being received via filter/store to do the p2p reliability logic.

@weboko
Copy link
Collaborator Author

weboko commented Aug 21, 2025

@fryorcraken good point, we indeed need only hash of the message and it is available without having an instance of Decoder

but, to accommodate for it in SDK we would need to expose it in Filter core which will make internals exposed - not a big problem per se but may lead to some misuse in future

and, separately, I think combining Encoder and Decoder just makes sense regardless of point above

  • these two entities often come together
  • combining them doesn't increase complexity of API or runtime
  • it is the same abstraction over proto anyway

and having that will help us avoid exposing Filter core (side effect)

Copy link
Member

@adklempner adklempner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice

@fryorcraken
Copy link
Collaborator

fryorcraken commented Aug 28, 2025

It is hard to value the merits of an API by just looking at it. Write a quick doc and mini app to demo so we can better judge.

How does it look like if Alice and Bob are sending each other messages, and they are both using ECIES (Alice has Bob's pubkey, Bob has Alice's), using your codec?

Edit: I see you crossed out "dogfooding". I would say dogfooding is mandatory when introducing a new API.

@weboko
Copy link
Collaborator Author

weboko commented Sep 5, 2025

agree, I have another PR with this entity there, will combine these PRs

@weboko weboko marked this pull request as draft September 5, 2025 22:25
@weboko
Copy link
Collaborator Author

weboko commented Sep 29, 2025

moving everything from here into main PR

#2583

@weboko weboko closed this Sep 29, 2025
@weboko weboko deleted the weboko/codec branch September 29, 2025 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: introduce Codec entity

4 participants